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Chapter 


Introduction 


What’s  is  the  Operational  Test  and  Evaluation 
Support  Tool  (OPS  TEST)? 

The  Operational  Test  and  Evaluation  Support  Tool  (OPS  TEST)  is  an  automated 
repository  of  training  and  evaluation  test  items  and  tests  that  was  developed  by 
the  United  States  Air  Force  Academy’s  Institute  for  Information  Technology 
Applications  (IITA).  It  is  made  freely  available  to  Department  of  Defense  and 
other  US  Government  agencies.  This  volume  of  OPS  TEST  documentation 
provides  information  on  using  OPS  TEST  for  taking  tests  on-line.  Specific  test¬ 
taking  functions  provided  are: 

•  Ability  of  crew  members  to  take  formal  training  and  evaluation  tests  on-line 
with  automatic  grading  and  feedback  to  the  test-taker. 

•  Ability  of  crew  members  to  build  on  demand  and  take  recurring  training  tests 
on-line  with  automatic  grading  and  feedback  to  the  test-taker. 

•  Ability  of  crew  members  to  access  the  Master  Question  Files  and  build  and 
take  practice  tests  from  these  files  with  immediate  feedback  on  each  question. 

Intended  Audience 

This  manual  is  intended  for  the  following  audience: 

•  Crew  Members  Taking  Tests:  Member  of  the  operations  crews  will 
use  OPS  TEST  to  take  their  formal  training  and  evaluation  tests  as  well  as 
to  access  the  Master  Question  Files  on-line. 

System  Requirements 

OPS  TEST  is  an  Access  97  application  intended  to  mn  within  a  Microsoft  Office 
97  environment  to  include  an  installed  version  of  Access  97.  OPS  TEST  requires 
a  Windows  95  capable  personal  computer,  preferably  a  Pentium  (or  faster)  with  at 
least  1 6MB  of  main  memory  (RAM)  and  3MB  of  available  disk  space.  A  monitor 
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running  with  a  resolution  of  at  least  800  X  600  is  also  assumed.  Using  a  lower 
resolution  will  result  in  some  forms  not  being  fully  displayed  and  accessible. 

Overview 

The  OPS  TEST  consists  of  two  primary  areas:  Administration  and  Test-Taking. 
This  manual  is  specifically  designed  just  for  the  test-taking  component.  A 
separate  OPS  TEST  Administrator’s  Manual  is  available  for  information  on  the 
full  range  of  OPS  TEST  capabilities  and  its  configuration. 

The  crew  member  test-taking  function  is  accessed  via  the  Windows’ 

START  |  PROGRAMS  |  OPS  TEST  TEST-TAKING  menu.  The  main  test¬ 
taking  menu  is  shown  as  Figure  1.1.  The  first  button  provides  access  to  the 
formal  training  and  evaluation  specific  tests  created  for  crew  members.  These 
tests  are  uniquely  build  for  individual  crew  members  and  are  password  protected 
for  security.  When  taking  these  tests,  crew  members  are  presented  one  test  item 
at  a  time  and  are  allowed  to  move  back  and  forth  through  the  entire  test 
answering  or  not  answering  test  items  as  they  prefer.  Specific  buttons  provide 
quick  access  to  un-answered  test  items.  When  the  test-taker  has  completed  the 
test,  they  press  a  button  to  have  it  graded.  If  they  have  any  un-answered  test 
items,  they  are  given  the  opportunity  to  go  back  and  answer  them  prior  to 
grading.  Once  the  test-taker  has  the  test  graded,  they  are  told  their  score  and 
given  the  opportunity  to  review  any  test  items  they  missed.  This  review  shows 
them  both  their  answer  and  the  correct  answer  together  with  the  associated 
reference  and  feedback  information  for  the  test  item.  Taking  formal  tests  is 
documented  in  detail  in  Chapter  3. 

The  second  option  provides  access  to  training  tests  that  are  designed  to  be 
created  upon  request.  These  tests  have  more  flexibility  in  that  they  do  not  need 
to  be  built  for  specific  individuals  ahead  of  time  and  may  be  set  to  be  re-entered 
multiple  times.  These  training  tests  are  documented  in  detail  in  Chapter  4. 

The  third  test-taking  option  is  for  crew  members  to  access  the  Master  Question 
File  (MQF)  on-line.  In  this  function,  they  are  provided  a  query  form  to  use  in 
selecting  test  items  from  the  MQF.  They  then  answer  these  questions  one  at  a 
time  and  are  provided  immediate  feedback  as  to  whether  or  not  they  correctly 
answered  the  test  item.  If  they  answered  incorrectly,  they  are  provided  both 
associated  references  and  the  item’s  feedback  information.  They  are  also 
provided  a  final  summary  of  their  results  to  include  an  overall  score  for  that  set  of 
test  items.  Detailed  documentation  on  MQF  test-taking  is  provided  in  Chapter  5. 
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Figure  1.1:  Main  Test-Taking  Menu 
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Chapter 


Installation  Procedures 


How  do  l get  the  OPS  TEST  Test-Taking  up  and  running \? 


OPS  TEST  Test-Taking  is  a  simple-to-install  Access  97  application.  The 
installation  consists  of  a  relatively  small  program  loaded  on  each  PC  that  will  be 
used  for  test-taking.  It  automatically  links  to  the  OPS  TEST  database  of  tests  and 
test  items  which  is  maintained  by  the  OPS  TEST  Administration  software 
documented  in  a  separate  manual. 

These  instructions  assume  that  you  have  the  OPS  TEST  database  installed  and 
configured  on  your  network.  If  this  is  not  the  case,  you  need  a  testbank 
administrator  to  first  perform  the  initial  database  installation  and  configuration 
procedures  described  in  Chapter  10  of  the  OPS  TEST  Administrator’s  Manual. 

OPS  TEST  Test-Taking  Installation 

Installation  of  OPS  TEST  Test-Taking  can  be  accomplished  using  the  installation 
compact  disk  (CD).  To  accomplish  installation  of  OPS  TEST  Test-Taking  via 
the  CD  use  the  instructions  below: 

1.  Place  the  OPS  TEST  Test-Taking  installation  CD  into  your  CD-ROM  drive. 

2.  Select  START  |  RUN  from  Windows  and  enter:  “d:\TestTaking\Setup.exe” 
(replace  the  “d:”  as  appropriate  if  your  CD-ROM  uses  a  different  device 
letter). 

3.  Select  all  the  default  options  presented  to  you. 

4.  When  you  initially  start  up  the  OPS  TEST  Test-Taking  program,  you  will  be 
prompted  for  the  location  of  the  OPS  TEST  Backend  Database.  Your  OPS 
TEST  Administrator  will  provide  you  with  this  path/location.  You  may  also 
be  prompted  for  a  password  for  the  OPS  TEST  Backend  Database.  This  also 
can  be  provided  by  your  OPS  TEST  Administrator. 
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Formal  Test  Taking 


Hmv  do  crew  members  take formal  training  and  evaluation  tests  on-line ? 

Formal  training  and  evaluation  tests  can  be  taken  on-line  using  the  Specific  Tests 
created  by  Test  Administrators.  These  tests  are  specific  to  the  crew  member  taking 
the  test  and  have  a  password  protection  mechanism.  Once  the  test  administrator  has 
created  the  test,  they  will  give  the  test-taker  the  password.  The  test-taker  will  access 
the  test  via  the  instructions  below  and  work  through  the  test  items  in  the  test.  When 
done  with  the  test,  the  test-taker  clicks  on  the  a  Grade  Test  button  at  which  point  the 
test  is  graded  and  the  test-taker  is  given  the  opportunity  to  review  and  get  feedback  on 
any  test  item  that  they  may  have  missed.  If  a  test  is  interrupted  once  it  has  been 
started,  the  test-taker  may  not  be  able  to  get  back  into  the  test  without  the  intervention 
of  the  Test  Administrator  to  “reset”  the  test.  This  depends  on  whether  the  “Can  Re- 
Enter?”  box  was  checked  for  the  test.  Regardless,  if  the  test  has  been  graded,  the 
Test-Taker  cannot  re-enter  it  without  the  test  administrator’s  assistance. 

Starting  a  Formal  Test 

Once  a  crew  member  to  be  tested  has  been  given  the  password  to  their  test  they  can 
access  the  main  Test-Taking  menu  shown  in  Figure  3.1. 
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Figure  3.1:  Open  a  Specific  Formal  Test 

This  can  be  accessed  by  the  START  |  PROGRAMS  |  OPS  TEST  TEST-TAKING 
menu  from  Windows. 

The  test-taker  then  clicks  on  the  Take  a  Formal  Training/Evaluation  Test  button. 
This  will  cause  a  Begin  Test  form  to  come  up  as  shown  in  Figure  3.2.  You  can  then 
locate  the  “Test”  in  the  pull-down  menu,  enter  the  provided  password,  and  click  the 
Begin  Test  button. 


Figure  3.2:  Selecting  a  Specific  Formal  Test  to  Open 

Once  the  test  has  been  successfully  entered  you  will  get  the  individual  test  items 
displayed  one  at  a  time  via  the  Crew  Member  Test-Taking  form  shown  in  Figure  3.3. 
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Answering  Formal  Test  Items 

To  answer  a  test  item  in  a  formal  test,  click  on  the  appropriate  radio  button  in  the 
Answer  Here  selection  box  in  the  lower  left  comer  of  the  form.  You  can  move  to  the 
next  test  item  using  either  of  the  right  arrow  buttons  provided  on  the  form.  The  first 
right  arrow  button  (with  the  shadowed  effect)  takes  you  to  the  next  unanswered  test 
item.  The  plain  right  arrow  takes  you  to  the  next  test  item  irregardless  of  whether  or 
not  it  has  been  already  answered.  The  left  pointing  arrow  move  you  to  previous  test 
items  and  the  one  with  the  shadowed  effect  moves  you  to  the  next  previous  test  item 
that  has  not  yet  been  answered.  If  the  test  item  has  an  associated  multimedia  clip,  the 
Play  Multimedia  button  will  be  available.  Click  on  this  button  to  have  the 
multimedia  clip  played  for  your  viewing.  You  may  play  the  clip  as  many  times  as  you 
like. 


Figure  3.3:  Formal  Test  Taking  Form 

Once  you  have  completed  the  test,  click  on  the  Grade  Test  button.  If  you  have  left 
any  test  items  unanswered  it  will  warn  you  and  provide  you  a  chance  to  go  back  and 
answer  them  prior  to  grading.  Otherwise  your  test  will  be  scored  and  your  score 
presented  to  you.  If  you  missed  any  test  items,  you  will  be  given  the  opportunity  to  go 
back  and  review  them. 

When  reviewing  missed  test  items,  you  will  not  be  able  to  change  your  answers  but 
you  can  use  the  arrows  to  move  through  the  missed  items.  With  each  missed  test  item 
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you  will  be  provided  the  governing  reference  for  that  item  and  perhaps  some 
additional  feedback  information  about  the  test  item. 

Completing  Formal  Tests 

After  the  test  has  been  graded,  per  the  instructions  above,  and  any  missed  items 
optionally  reviewed,  simply  close  out  of  the  remaining  forms  to  exit  OPS  TEST  Test- 
Taking. 
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Chapter 


Monthly  Training  Tests 

Hoip  doyou  take  the  Monthly  training  Tests ? 


Crew  members  can  use  OPS  TEST  Test-Taking  to  take  monthly  training  tests  that  are 
built  for  them  by  their  training  personnel.  These  tests  are  built  upon  request  and  some 
can  be  closed  without  being  graded  and  then  later  re-entered  with  all  the  previous 
responses  kept  intact.  The  status  and  results  of  these  tests  are  typically  monitored  by 
and  reported  to  the  training  personnel. 

Taking  a  Monthly  Training  Test 

When  a  crew  member  wants  to  take  a  monthly  training  test,  they  can  do  so  by  clicking 
on  the  Take  Monthly  training  Test  button  of  the  main  Test-Taking  form  (seen  as 
Figure  1.1).  This  can  be  accessed  via  the  START  |  PROGRAMS  |  OPS  TEST-TEST 
TAKING  menu  from  Windows. 

Clicking  on  the  Take  Monthly  Training  Test  button  on  this  main  Test-Taking  form 
brings  up  a  form  that  allows  the  crew  member  to  specify  the  test  they  wish  to  take, 
their  name,  and  the  password  for  the  test  (if  one  has  been  placed  on  the  test).  This 
same  form  is  used  to  re-enter  tests  that  were  started  but  not  completed.  This  form  is 
shown  in  Figure  4.1  below.  Once  this  information  has  been  specified,  click  on  the 
associated  Take  Test  or  Re-Enter  Test  button  to  access  the  test. 
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Figure  4.1:  Selecting  MQF  Test  Items 

Answering  Monthly  Training  Test  Questions 

Once  you  have  gained  access  to  the  training  test,  the  test  items  are  displayed  one  at  a 
time  using  the  form  shown  in  Figure  4.2  below.  If  there  is  a  multimedia  clip 
associated  with  the  test  item,  the  Play  Multimedia  button  will  be  shown  and  you  can 
click  on  it  to  play  the  clip  as  many  times  as  you  like.  Answer  the  test  items  by  selecting 
the  appropriate  response  in  the  Answer  Here  selection  box  in  the  lower  left  corner  of 
the  form. 

As  with  formal  testing,  you  can  move  freely  between  test  items  using  the  arrow 
buttons.  The  ones  with  the  shadows  move  to  the  previous/next  un-answered  item 
while  the  plain  arrows  move  you  to  the  immediate  prior/ next  item. 

Once  you  have  answered  all  the  questions,  you  can  click  on  the  Grade  Test  button  to 
have  your  test  scored.  If  you  answered  any  questions  incorrecdy,  you  will  be  provided 
the  opportunity  to  review  them  with  the  correct  answer  and  some  feedback  text 
shown  for  you.  Some  test  items  also  include  hyperlinks  to  associated  reference 
material. 

If  you  wish  to  exit  the  test  prior  to  completing  it,  you  can  click  on  the  Close  Form 
button.  Depending  on  how  the  test  was  created,  you  may  or  may  not  be  able  to  re¬ 
enter  it  later.  If  this  option  is  not  available  on  your  test,  you’ll  need  to  have  your 
training  test  administrator  reset  the  test  for  you. 
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Figure  4.2:  Answering  Test  Items 
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Chapter 


On-Line  Access  to  Master 
Question  File 

Hoiv  dojou  access  tlx  Master  Question  File  to  take practice  tests  on-line ? 


Crew  members  can  use  OPS  TEST  to  call  up  practice  test  items  from  the  Master 
Question  File  (MQF).  Practice  test  items  are  retrieved  via  a  query  form.  The  crew 
member  may  answer  as  many  or  as  few  of  the  retrieved  test  questions  as  they  wish. 
None  of  their  responses  will  be  recorded  or  can  be  traced  back  to  them.  As  each 
retrieved  MQF  test  item  is  answered,  the  test-taker  is  given  immediate  feedback  as  to 
whether  they  answered  correctly  or  not.  If  they  did  not  correctly  answer,  the  test  item 
is  displaced  with  some  additional  reference  information  (including  the  correct  answer) 
so  they  can  learn  the  objective  of  the  question.  Upon  completing  a  set  of  retrieved 
questions,  an  overall  score  for  that  set  of  test  items  is  provided.  At  this  point  the  crew 
member  can  either  retrieve  some  more  MQF  test  items  or  exit. 

Selecting  MQF  Questions 

When  a  crew  member  wishes  to  answer  some  practice  test  items  from  the  MQF,  they 
can  access  the  main  Test-Taking  form  (seen  as  Figure  1.1).  This  form  can  be  accessed 
via  the  START  |  PROGRAMS  |  OPS  TEST  TEST-TAKING  menu  from  Windows. 

Clicking  on  the  Practice  Test  from  the  MQF  button  on  this  main  Test-Taking  form 
brings  up  a  query  form  that  can  be  used  to  select  questions  from  the  MQF  (see  Figure 
5.1  below).  Several  of  the  selection  fields  allow  the  use  of  wildcards  (“?”  and  “*”)  to 
substitute  for  one  or  many  of  any  characters.  You  can  also  find  current  values  for 
these  fields  using  the  pull-down  lists  provided.  All  specified  criteria  must  be  met  by 
the  test  items  returned.  Clicking  on  the  Take  Practice  Test  button  will  retrieve  the 
selected  test  items  and  display  them  one  at  a  time  to  be  answered. 
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Figure  5.1:  Selecting  MQF  Test  Items 

Answering  MQF  Questions 

Once  test  items  have  been  retrieved  from  the  MQF  via  the  query  form  described 
above,  they  are  displayed  one  at  a  time  using  the  MQF  Test  Taking  form  shown  in 
Figure  5.2  below.  If  a  multimedia  clip  is  associated  with  the  test  item,  the  Play 
Multimedia  button  will  be  shown  and  you  can  click  on  it  as  many  times  as  you  want 
to  play  die  clip.  Answer  the  test  items  by  selecting  the  appropriate  response  in  the 
Answer  Here  selection  box  in  the  lower  left  corner  of  the  form.  Your  response  will  be 
immediately  checked  against  the  correct  answer  and  you  will  be  told  if  you  got  it  right. 
If  you  missed  it,  you  will  be  provided  some  feedback  information  on  the  question  and 
what  the  correct  answer  is.  You  will  also  be  provided  with  a  hyperlink  to  an  associated 
reference  for  some  test  items.  Clicking  on  the  link  will  open  that  reference  up  for  you. 

Once  you  have  answered  all  the  questions,  you  will  be  given  a  summary  score  for  that 
set.  You  can  then  select  more  questions  or  simply  exit  out  of  the  form(s)  via  the  open 
door  buttons  in  the  lower  right  corner.  None  of  your  responses  to  MQF  questions 
will  be  recorded  or  are  trace-able  back  to  you. 


Figure  5.2:  Answering  MQF  Test  Items 
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